Turbo decoder stopping based on mean and variance of extrinsics

ABSTRACT

An adaptive stopping criterion improvement for an iterative decoder, in which iteration is halted by observing a signal-to-noise ratio generated from the mean and variance of the soft estimates (or their divergence).

BACKGROUND AND SUMMARY OF THE INVENTION

[0001] The present application relates to error control systems in datatransmission, especially to iterative decoding using parallelconcatenated encoders, such as turbo decoders.

[0002] Background: Signals and Noisy Channels

[0003] No communications channel is perfect. In the real world, everycommunications channel includes some “noise” (unpredictable non-signalinputs). The amount of noise is usually quantified by thesignal-to-noise ratio, or “SNR”, which is stated in decibels. (Forexample, if the signal power is one hundred times the noise power, theSNR would be about 20 dB.) Higher SNRs are more desirable, but the SNR(at a given transmission rate) is set by the physics of the channel, aswell as by the design of the transmitter and receiver. Even thoughdigital signals are generally less susceptible to degradation thananalog signals, many channels include enough noise to induce some errorsin a digital signal. In many applications a desired data rate can easilybe achieved at reasonable cost, but the worst-case level of bit errorswith such a configuration is excessive. In such cases coding techniquescan be used to reduce the error rate (at the cost of a slightly reducedgross data rate).

[0004] Similar techniques can also be used with data storagearchitectures. In this case, the “channel” is the storage medium and theinterfaces to it. For example, in a hard disk drive the write head is ananalog driver stage which may not be perfectly aligned to the desiredradial position on the disk. Data can be degraded, during writing,storage, or reading, by factors such as mistracking, overtemperature,particulate contamination, or mechanical failure.

[0005] Background: Error-Control Coding

[0006] Coded digital communication systems use error control codes toimprove data reliability at a given signal-to-noise ratio (SNR). Forexample, an extremely simple form (used in data storage applications) isto generate and transmit a parity bit with every eight bits of data; bychecking parity on each block of nine bits, single-bit errors can bedetected. (By adding three error-correction bits to each block,single-bit errors can be detected and corrected.) In general, errorcontrol coding includes a large variety of techniques for generatingextra bits to accompany a data stream, allowing errors in the datastream to be detected and possibly corrected.

[0007] A famous existence proof (Shannon's noisy channel coding theorem)states that error rates can be made as low as desired by using optimalerror control codes. This theorem did not itself tell how to derive theoptimal error control coding, and for decades no coding schemes wereable to come close to the theoretical limits defined by this theorem.However, a major breakthrough was achieved in 1993, when turbo codeswere introduced.

[0008] Background: “Turbo” Coding

[0009] The encoder side of a turbo coding architecture typically usestwo encoders, one operating on the raw data stream and one on a shuffledcopy of the base data stream, to generate two parity bits for each bitof the raw data stream. The encoder output thus contains three times asmany bits as the incoming data stream. This “parallel concatenatedencoder” (or “PCE”) configuration is described in detail below.

[0010] The most surprising part of turbo coding was its decodingarchitecture. The decoder side invokes a process which (if the channelwere noiseless) would merely reverse the transformation performed on theencoder side, to reproduce the original data. However, the decoder sideis configured to operate on soft estimates of the information bits andrefines the estimates through an iterative reestimation process. Thedecoder does not have to reach a decision on its first pass, but isgenerally allowed to iteratively improve the estimates of theinformation bits until convergence is achieved. (A more detaileddescription of this is provided below.)

[0011] One drawback to the turbo decoder is that some received codewordsrequire many iterations to converge. The ability to improve estimates byiteration is a great strength, since it means that the decodingperformance can be nearly optimal; but every iteration requiresadditional time, computing resources, and/or battery energy. Thus inpractice it is desirable to find a way to stop turbo decoding as soon asconvergence is achieved. Worst-case channel conditions may require manymore iterations than do best-case conditions, and it is desirable tofind some way to adjust the number of iterations to no more thanrequired.

[0012] Background: Stopping Criteria for Turbo Decoders

[0013] Originally, turbo decoders would execute a specific number ofiterations regardless of the number of errors inserted by the channel.This was inefficient, since fewer iterations are needed when there arefewer errors in the incoming signal. Efforts have therefore been made todevelop adaptive stopping criteria, which would provide an indicator ofwhen the turbo decoding iterations can be stopped.

[0014] U.S. Pat. No. 5,761,248 describes adaptive stopping based oncross entropy, or convergence of the output bits: when the decoder'sdigital output bit estimates (for some block of data) stop changing, thedecoding process for that block is halted. However, this stoppingcriterion requires at least two iterations of the decoding process (inorder to generate enough outputs to compare to one another). This can bea waste of processing effort in systems where the channel introduces fewerrors, or where there is a large amount of data to be decoded. Otherstopping criteria, based on the cross entropy criterion, abort decodingbased on the ratio of sign changes in the extrinsics to the number ofextrinsics.

[0015] Another attempt at adaptive stopping was described in Robertson,“Illuminating the Structure of Code and Decoder of Parallel ConcatenatedRecursive Systematic (Turbo) Codes,” 1994 GLOBECOM Proceedings 1298,which generates an estimate of the decoding quality from the secondmoment of decoded bits.

[0016] Turbo Decoder Stopping Based on Mean and Variance of Extrinsics

[0017] The present application teaches innovative architectures andmethods for aborting the decoding iteration process. Instead of acomparison between successive outputs of the MAP decoders, an absoluteindication of the decoded signal quality is compared with a thresholdsignal quality. The decoders generate soft outputs, or estimates of theoriginal transmitted data symbols. With each decode, these extrinsicvalues diverge (or their absolute value increases), which indicatesgreater certainty of the value of the transmitted bit. The values of theextrinsics are used to compute an estimate of the overall signalquality. This estimate of signal quality is constructed from both themean of output extrinsics and the variance of the extrinsics. When thesignal quality reaches the desired performance threshold, iterations areaborted.

BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The disclosed inventions will be described with reference to theaccompanying drawings, which show important sample embodiments of theinvention and which are incorporated in the specification hereof byreference, wherein:

[0019]FIG. 1 shows a block diagram for parallel encoders.

[0020]FIG. 2 shows a block diagram for a recursive systematic encoder.

[0021]FIG. 3 shows a block diagram of a turbo decoder.

[0022]FIG. 4 shows a stopping criterion implementation using division bya variable quantity N.

[0023]FIGS. 5a and 5 b show graphs of SNR calculated from soft-decisionoutput values as MAP decodes are executed.

[0024]FIG. 6 shows a stopping criterion implementation without divisionby N.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] The numerous innovative teachings of the present application willbe described with particular reference to the presently preferredembodiment. However, it should be understood that this class ofembodiments provides only a few examples of the many advantageous usesof the innovative teachings herein. In general, statements made in thespecification of the present application do not necessarily delimit anyof the various claimed inventions. Moreover, some statements may applyto some inventive features but not to others.

[0026] Additional Background

[0027] Coded digital communication systems use error control codes toimprove data reliability. Both the physical propagation medium (thechannel) and the hardware used to receive a signal introduce error inthe form of noise or signal degradation.

[0028] Decoding makes use of redundant bits added to a data stream todetect errors and estimate the most probable value for data bits giventhe other bits received. The effectiveness of error control codes isexpressed in terms of coding gain, which is the difference between theEb/No needed to achieve a given bit error rate (BER) in a coded systemand the Eb/No required to achieve the same BER in an uncoded system.

[0029] Though Shannon's noisy channel coding theorem states thatarbitrarily low BERs are possible using error control codes, no codingschemes were able to achieve gain near the Shannon limit until 1993 whenturbo codes were introduced. Turbo coding provides error controlperformance to within a few tenths of a decibel of the Shannon limit.

[0030] The use of parallel concatenated encoders and iterative decodersdistinguishes turbo codes from other coding systems. A PCE consists oftwo or more recursive systematic encoders (either block or convolutionalencoders). Encoders add parity bits to the original message in order todetect the presence of errors in the message. (In a rate ½ convolutionalencoder, for instance, two data bits are output for every input databit.) In the case of two convolutional encoders (a typical case forturbo code schemes) an original message block is encoded using the firstencoder, generating a first codeword. The original message is theninterleaved (i.e., the input data symbols are reproduced in a differenttemporal order) and the result is encoded by the second encoder,producing a second codeword. The original message and both codewords arethen multiplexed and transmitted to the receiver. The component encodersare typically encoders for block codes or convolutional codes. Thesymbols are permuted either on a block basis (a block interleaver) or ona sliding window basis (a convolutional interleaver).

[0031] Turbo decoders work by passing soft decoding information betweentwo decoding algorithms. Each decoder concentrates on one side of theencoding process, and through successive computations the two decodersarrive at a consensus on the detection of the transmitted data. Onedecoder executes and generates N new extrinsics (a posterioriprobabilities for the value of a given information symbol, where N isthe number of data symbols), which are then applied to the otherdecoder. This second decoder executes and generates N new extrinsics,which are passed back to the first decoder. This completes one iterationof the decoding process.

[0032]FIG. 1 shows a typical encoder setup. The original data stream, x,is encoded by two recursive systematic convolutional encoders 100, 102.(An encoder is systematic if it sends the uncoded data as part of thecoded message, as shown in the rate-½ recursive systematic encoderdiagram of FIG. 2.)

[0033] In the encoding process, x is encoded by the top encoder 100, andx and p₁ (the first codeword) are output. The data x is also interleavedby an interleaver 104 and input to the bottom encoder 102, whichgenerates outputs x* and the second codeword, p₂. The original data x,p₁, and p₂ are combined by combiner circuitry 106 and sent through thechannel to the receiver. (Note that the interleaved x* is not sent,since it is redundant information.)

[0034]FIG. 2 shows a recursive systematic convolutional encoder 200. Thebinary data stream x is fed into a shift register circuit 202 consistingof a series of memory elements. With each successive input, the memoryelements are added according to a fixed pattern, creating a pair ofoutputs. One output is the unchanged data x, the other is the coded datap. These outputs can be multiplexed to create a single coded datastream.

[0035] Decoders for PCEs would be expensive and computationallyinefficient without iterative decoding. Iterative decoding, or turbodecoding, is a relatively low complexity decoding method that exploitsthe component substructure of the PCE. An iterative decoder has asoft-input/soft-output (SISO) component decoder for each componentencoder of the PCE. The decoders generate separate a posterioriprobabilities for a given information symbol. The decoders take turnsdecoding the received data, operating on each other's incompletelydecoded output. With each iteration, the estimation of the value of aninformation symbol improves in accuracy.

[0036]FIG. 3 shows a block diagram of a turbo decoder 300. The originalmessage block x and code words p₁ and p₂ are scaled by 2/σ² (a measureof the channel noise variance) and stored in three separate memories ofsize N as ^(Λ)(x), ^(Λ)(p₁), and ^(Λ)(p₂) respectively. Interleavers302, 304 for the data are also of size N. The extrinsic memories haveoutputs W₁ and W₂ (the a posteriori probability data, or estimates ofthe probability distribution of a code symbol that are updated with eachiteration) which are set to the initial conditions before the firstdecoding iteration. The scaled inputs ^(Λ)(x) and ^(Λ)(p₁) and thedeinterleaved W₂ are applied to the upper MAP decoder 308, as shown inthe diagram. This MAP decoder 308 executes and generates N newextrinsics W₁ which are stored in the extrinsic memory within theinterleaver I 304.

[0037] Interleaved ^(Λ)(x), ^(Λ)(p₂), and interleaved W₁ are applied tothe lower MAP decoder 310. The lower MAP decoder 310 executes andgenerates N new extrinsics W₂ which are stored in the extrinsic memoryof the deinterleaver I⁻¹ 306. This completes the first decodingiteration. At each iteration, the upper MAP decoder 308 operates on^(Λ)(x), ^(Λ)(p₁), and the deinterleaved W₂ (which is updated at eachiteration). The lower MAP decoder 310 operates on interleaved ^(Λ)(x),^(Λ)(p₂), and the interleaved W₁ (which is also updated at eachiteration).

[0038] Iterations are executed until a specified number have beencompleted or until the iterations are aborted. When decoding iscomplete, ^(Λ)(x), W₁, and deinterleaved W₂ are summed. The result issliced, meaning the same component of each vector ^(Λ)(x), W₁, anddeinterleaved W₂ are used to determine that component of the outputvector x, which is the decoded message. These decoded bits are thenoutput.

[0039] The number of iterations performed varies, typically ranging fromthree to twenty. Depending on the noise pattern of a given receivedcodeword, the decoder might reach its optimum solution after only one ortwo iterations. The remaining iterations would not improve performanceand are a waste of processing power and time. If the noise pattern of areceived codeword is severe, the decoder may require many iterations toreach an optimum solution.

[0040] Soft-Decisions

[0041] In digital communication systems, data is sent in the form ofsymbols (usually binary) that take on one value of a finite set ofpossible values. These symbols are modulated onto a carrier signal thatis transmitted through the channel to a receiver. In the receiver,detection circuitry examines the received signal and determines whatsymbols were most likely transmitted. For example, in a binaryhard-decision system, the receiver determines if a zero or a one wassent, and outputs a zero or one accordingly. No information as to thequality of the symbol determination is included in a hard-decisionoutput.

[0042] Soft-decision decoding takes advantage of additional informationgenerated by the receiver decision circuitry. Rather than assign a zeroor a one to each received binary signal, soft-decision receivers use anexpanded set of symbols to describe the value of a given transmittedpiece of data. For example, in one implementation, each binary datasymbol transmitted in the original data stream is expressed as aneight-bit value. Different regions are established that indicate thestrength of the decision, with intermediate values given to decisionsthat are less reliable.

[0043] In the preferred embodiment, the output of the encoder is binary(zeros and ones). A modulator maps the ones to a positive one value andmaps the zeros to a negative one value. When the receiver determines aparticular received symbol, it expresses it as an eight-bit value, the“soft” decision. The closer a soft-decision gets to its extreme value,the more reliable the estimation of the original data bit. Theseestimates are made after each MAP decode, which is a substep of a fulliteration. In the preferred embodiment, one iteration consists of twoMAP decodes. (Note that the number of MAP decoders required depends onthe number of encoders used.)

[0044] As a turbo decoder goes through its iterations, the certainty ofthe value of a given information symbol generally increases. Thus, witheach iteration substep (i.e., each MAP decode), the value of a “zero” inthe original data stream will become more and more negative (since thezeros are modulated into negative numbers) and the value of a “one” inthe original data stream will become more and more positive (because theones were modulated as positive numbers).

[0045] Stopping Criteria for Turbo Decoders

[0046] All the functions of the stopping criterion can be easilyimplemented in hardware except for division, which cannot be implementedin an efficient manner. Typically, division functions like A/B areimplemented as A×1/B. 1/B is implemented as a lookup table. Thissolution works satisfactorily as long as the range of numbers for B issmall. In a previous method of turbo decoding, the stopping criterionrequired division by a variable quantity N. N (the number of extrinsicsgenerated by the component decoders) can be large, ranging from 320 to5120. Often, division by N must be performed from 10² to 10⁶ times. Itis therefore inefficient to implement this one division.

[0047] Turbo Decoder Stopping Criterion Improvement

[0048] The present application discloses an innovative improvement tothe turbo decoding stopping criterion. The improved stopping criterionrequires no division functions. By manipulating the algorithm, divisionby message dependent quantity N is eliminated and replaced withmultiplication by a constant. This greatly simplifies the execution ofthe algorithm and makes hardware implementation much more efficient.

[0049]FIG. 4 shows a block diagram of a stopping criterionimplementation. The block diagram is broken into two regions. The tophalf generates the two moments M₁ and M₂, which are needed to computethe SNR for the extrinsics. M₁ is the sum of the N extrinsics at anygiven point in the iterative decoding process, and M₂ is the sum of thesquares of the N extrinsics at a given point. Their equations are

M₁=Σ|W_(i)|

[0050] and

M₂=Σ|W_(i)|²

[0051] where i is summed from 1 to N. The top section is executed oncefor each of the N extrinsics, generating a single M₁ and a single M₂.Each component of the extrinsic W_(i) is input and added to the sum,which is stored in the register.

[0052] M₁ and M₂ serve as input for the bottom half of the stoppingcriterion. These quantities are used to compute a signal-to-noise ratioestimate from the extrinsics. The extrinsics W_(i) are assumed to bedistributed as a bimodal Gaussian probability distribution function,with one mean at +m and another mean at −m. Each mode has the sameconditional variance v². The absolute value of the probabilitydistribution function is taken, making a single mean at +m. An increasein this mean indicates greater certainty in the value of the decodedbits. An increase in the variance would indicate less certainty in thedecoded values.

[0053] Alternatively, the divergence of the decoded bits can be used. Ifthe absolute value is not taken of the probability distributionfunction, the two means (the positive and negative modes) will divergeas decoding progresses.

[0054] An estimate of signal quality based on the mean and variance ofthe extrinsics is computed by dividing the m² (the signal) by v² (thenoise). The mean m is estimated by $m = \frac{\sum\quad {W_{i}}}{N}$

[0055] and the optimal estimate for the variance is$v^{2} = {\frac{\sum\quad {W_{i}}^{2}}{N} - \left( \frac{\sum\quad {W_{i}}}{N} \right)^{2}}$

[0056] The bottom half of the block diagram in FIG. 4 generates thesignal quality estimate, which is then compared with the threshold K. Ifthe quality estimate is greater than the threshold, then the Exceedsignal is active. The Exceed signal will inform the turbo decoder tostop executing its remaining iterations because an optimum solution hasbeen reached. The bottom half executes only once per MAP decode.

[0057]FIGS. 5a and 5 b show a graphical representation of the change inextrinsic values as MAP decodes are executed. The “x” symbols representSNR values calculated for the extrinsics after each MAP decode. In FIG.5a, 5 MAP decodes are executed before the SNR of the extrinsics exceedsthe threshold. After the 5^(th) decode, no more iterations are executed.Note that this represents only 2.5 full turbo decode iterations, sincetwo MAP decodes complete one iteration in the preferred embodiment.

[0058]FIG. 5b shows an example decode for a system with fewer errorsthan shown in FIG. 5a. When fewer errors are inserted in the datastream, fewer MAP decodes are necessary to reach an acceptable SNR.Thus, as shown in FIG. 5b, only two MAP decodes produces a SNR above thethreshold. Decoding would stop at this point.

[0059] The following algebraic expression for the bottom half of thediagram shows the comparison of the SNR (computed from the extrinsics)with K, the threshold SNR.$\frac{\left( \frac{\sum\quad {W_{i}}}{N} \right)^{2}}{\frac{\sum\quad {W_{i}}^{2}}{N} - \left( \frac{\sum\quad {W_{i}}}{N} \right)^{2}} > K$

[0060] The top half of the block diagram requires 2N additions, Nmultiplications, and N absolute value functions. The bottom halfrequires 3 divisions, 1 multiplication, 1 subtraction, and 1 comparison.

[0061]FIG. 6 shows the innovative stopping criterion in a block diagram.The comparison equation for the bottom half of the algorithm isrewritten by multiplying the denominator and numerator by N². Theequation can be rewritten as follows.$\frac{\left( {\sum\quad {W_{i}}} \right)^{2}}{{N{\sum\quad {W_{i}}^{2}}} - \left( {\sum\quad {W_{t}}} \right)^{2}} > {{k\left( {1 + \frac{1}{K}} \right)} \times \left( {\sum\quad {W_{t}}} \right)^{2}} > {N{\sum\quad {W_{i}}^{2}}}$

[0062] This equation is the basis for the block diagram shown in FIG. 6.The bottom half division functions are replaced by two multiplications.One multiplier can be used and multiplexed three times for an efficientimplementation. The K threshold has now been replaced with a 1+1/Kthreshold. Since K is a constant, 1+1/K is also a constant which can beprecomputed by the user or generated from a simple lookup table prior tothe start of a turbo decode.

[0063] The top half of the algorithm is the same as the stoppingcriterion in FIG. 4. The bottom half of the diagram shows that N is nolonger a denominator, which greatly simplifies hardware implementation.M₁ is squared and multiplied by 1+1/K. This value is compared to theproduct of M₂ and N.

[0064] The value of the threshold is still a constant, and when thecomputed quality estimate of the decoded signal exceeds the threshold,the decoding iterations are stopped. This allows the decoding tocontinue when there are large numbers of errors introduced into thesignal by the channel while it prevents the decoder from executinguseless iterations when few errors are introduced.

[0065] Definitions:

[0066] Following are short definitions of the usual meanings of some ofthe technical terms which are used in the present application. (However,those of ordinary skill will recognize whether the context requires adifferent meaning.) Additional definitions can be found in the standardtechnical dictionaries and Journals.

[0067] Eb/No: Ratio of energy per modulating bit to the noise spectraldensity.

[0068] BER: Bit error rate.

[0069] PCE: Parallel Concatenated Encoder.

[0070] SI/SO: Soft-input soft-output. “Soft” input or output refers tousing greater bit size (for instance, 8-bits) to describe a given datasymbol.

[0071] MAP decoder: Maximum A-Posteriori. MAP decoders use a detectioncriterion that leads to the selection of x that maximizes theprobability p(x/r) of a symbol x given some received information r.

[0072] Extrinsic: Outputs of decoders that estimate the values ofdecoded bits. Extrinsics are usually soft estimates.

[0073] Modifications and Variations

[0074] As will be recognized by those skilled in the art, the innovativeconcepts described in the present application can be modified and variedover a tremendous range of applications, and accordingly the scope ofpatented subject matter is not limited by any of the specific exemplaryteachings given, but is only defined by the issued claims.

[0075] In particular, the disclosed inventions are not necessarilylimited to “turbo decoding” as implemented in 1999. This is a veryactive area of technological development, and many improvements andmodifications are undoubtedly in existence or soon to come. Even iffuture improved decoding architectures with iterative processing of softestimates are referred to by names other than “turbo decoding”, thedisclosed inventions may still be applicable.

[0076] For instance, multi-bit symbols could be used in lieu of binarysymbols as the original data stream. Alternatively, future versions ofiterative decoding might apply to analog signals rather than digitalsignals. The claimed inventions can also be applied to decoderarchitectures that implement these changes.

[0077] Other decoding methods, such as pipeline decoding, or turbo codesthat use more encoders than described in the present application mayalso be within the contemplation of the present innovations.

[0078] Further details and descriptions of the present state of the artmay be found in TURBO CODING, by Heegard and Wicker, and ERROR CONTROLSYSTEMS, by Wicker, which are hereby incorporated by reference.

What is claimed is:
 1. A turbo decoding system, comprising: a decodermodule, using an adaptive abort criterion; wherein the adaptive abortcriterion is based on the mean and the variance of partially decodedextrinsics.
 2. The system of claim 1 , wherein the abort criterion isgenerated as a ratio of the mean and the variance of the extrinsics. 3.An iterative decoder system for a recursive systematic encoder,comprising: a decoder module, wherein estimates of data symbols aregenerated through an iterative decoding process; a comparison algorithmfor comparing a derived quality attribute of the generated data symbolestimates to a predetermined threshold; wherein said quality attributeis based on the mean and the variance of the estimates of the datasymbols; and wherein decoding is aborted based on the comparison result.4. The system of claim 3 , wherein the quality attribute is generated asa ratio of the mean and the variance of the estimates.
 5. A method fordetermining an abort criterion in turbo decoding, comprising the stepsof: generating extrinsic values; for each extrinsic value, generating asignal-to-noise ratio; comparing the generated signal-to-noise ratio toa threshold signal-to-noise ratio; and aborting based on the comparisonresult; wherein said signal-to-noise ratio is computed from the mean andthe variance of the extrinsics.
 6. The method of claim 5 , wherein thesignal-to-noise ratio is computed by dividing the mean of the extrinsicvalues by the variance of the extrinsic values.
 7. A method fordetermining an abort criterion in iterative decoding, comprising thesteps of: generating estimates of data symbols; generating a qualityattribute based on the mean and variance of the estimates; comparing thequality attribute to a predetermined threshold; aborting the turbodecoding based on the comparison result.
 8. The method of claim 7 ,wherein the quality attribute is generated as a ratio of the mean andthe variance of the estimates.
 9. A method for determining an abortcriterion in iterative decoding, comprising the steps of: generatingestimates of data symbols after an iteration substep; measuring the meanof the estimates; measuring the variance of the estimates; generating aquality attribute based on the mean and the variance; comparing thequality attribute to a predetermined threshold; and aborting the turbodecoding based on the comparison result.
 10. The method of claim 9 ,wherein the quality attribute is generated as a ratio of the mean andthe variance of the estimates.